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students gain a qualitative understanding of important physics 
concepts. The tutor simulates a teaching strategy called "bridging 
analogies" that previous research has demonstrated to be successful 
in one-on-one tutoring and written explanation studies. The strategy 
is designed to remedy misconceptions by appealing to existing correct 
intuitions, and extending these intuitions by encouraging analogical 
thinking. The strategy was motivating and successful for some 
learning situations. Analysis of the data indicates those situations 
in which the strategy worked well and those where alternatives are 
needed. Implications for improving the tutor using artificial 
intelligence technology to incorporate a representation of student 
beliefs and intelligent sequencing of example presentations are 
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Abstract 

We describe a computer tutor designed to help students gain a 
qualitative understanding of important physics concepts. The tutor 
simulates a teaching strategy called "bridging analogies" that previ- 
ous research has demonstrated to be successful in one-on-one tutoring 
and written explanation studies. The strategy is designed to remedy 
misconceptions by appealing to existing correct intuitions, and extend- 
ing these intuitions by encouraging analogical thinking. Students were 
videotaped while using the program ar.d were encouraged to think 
aloud. The strategy was motivating and successful for some learning 
situations. Analysis of the data indicates those situations in which the 
strategy worked well, and those where alternatives are needed. We 
ou "i • implications for improving the tutor using artificial intelligence 
technology to incorporate a representation of student beliefs and intel- 
ligent sequencing of example presentations. 
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1 Introduction 

In this paper we report on a computer simulation of a human tutoring 
strategy which previous cognitive studies have shown to be effective in one- 
on-one human tutoring. The purpose of the research has been to evaluate 
the strengths and limitations of an automated version of the strategy, which 
we envision as ultimately being one of several strategies employable by an 
intelligent computer tutor. First we will place our work in the context of 
previous ITS research efforts. 

Using artificial intelligence technology to build computer tutors can en- 
hance learning in two ways. First, expert systems techniques combined 
with sophisticated knowledge representation methods facilitate the creation 
of learning environments which simulate complex processes or reify abstract 
systems and concepts. Second s AI technology facilitates the representation 
of human teaching expertise and pedagogical domain information. It can 
provide personalized feedback and assistance which is strategically relevant 
to the discourse context and the topic's pedagogical idiosyncrasies. 

To date, most ITS research has focused on designing semantically rich 
learning environments or on the conceptual analysis ot topic areas and tasks. 
Though these efforts are important, this paper addresses the paucity of 
research dealing with the act of tutoring itself. Though many intelligent 
tutoring systems incorporate expertise in teaching, diagnosis, and commu- 
nication, few are explicit about the following: 1. what teaching strategies 
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are used; !>. assumptions made abou*. the nature of learning and teaching 
which underpin these strategics; and 3. the precise rules or algorithms that 
instantiate these strategics. Sharing this information is conducive to the 
comparative analysis of tutoring system descriptions. It also encourages the 
proliferation of research that ronfirms or extends the findings of research de- 
signed to test or evaluate tutoring systems. Notable exceptions to the above 
concerns nrc the research cfTorts of Anderson ct. al. (8i), who investigates 
the learning pioccss, Stevens & Collins (/7), and Littman ct. al. (8G), who 
investigate the tutoring process, and Clancey (82) and Woolf (84), who in- 
vestigate how teaching expertise can be explicitly represented in computer 
tutors. 

As mentioned above, the computer tutor described here has been de- 
signed to simulate a human tutoring strategy. The strategy remedies mis- 
conceptions by anpcaling to existing correct intuitions, and extending these 
intuitions by encouraging analogical thinking. An overview of the strategy 
is as fo!!o-vs. The strategy is invoked when the student is diagnosed to be 
harboring a misconceptions, as evidenced by giving an incorrect answer to a 
target problem. The tutor then attempts to find an "aijchor" situation. An 
anchor is a simple analogous situation for which the student has a correct 
intuition. For the domain we are investigating, an example target problem 
is the question of whether a table exerts an upward force on the book that 
rests on it. Many students 2xhibii the misconception that rigid objects can* 



not exert forces. A typical anchoring situation is a book being laid in a 
hand. Most students will agree that the hand exerts an upward force on 
the book. Typically, however, the student does -lot immediately see that 
the situations are truly analogous with respect to the questions being asked. 
Intermediate analogous situations, called "bridging analogies" are used to 
bridge the gap between correct intuitions and misconceptions. For example, 
a bridge between the book on the table situation and the book on the hand 
situation is a book on a spring. Bridging situations aic presented one at a 
time to the student, churn according to the student's previous responses, as 
described later. The student is asked to compare his answers about bridging 
situations with selected previous answers. These comparisons encourage the 
cognitive conflict that can motivate the student to see the problem situa- 
tion as truly analogous to the anchor. The strategy has proven successful in 
remediating physics misconceptions in recently conducted human tutoring 
and written explanation studies. 4 

The computer "tutor" described here is not an intelligent program. 8 It is 
not intended to be u.sed in a stand-alone fashion to teach. It is an encoding 
of a specific teaching strategy which research has shown to be elective in 
certain contexts. The purpose of the research is to evaluate the strengths 
and limitations of a com puter version of the strategy. For this reason we 

<The strategy hax also been field tested in high school clwes with encouraging result* 

(Clement 87, and Camp ct. »!. 87). 

8 Dut see Section 5 for implications for an intelligent version of the strategy 



have not embellished the program in ways which might make it teach better, 
if these embellishments would cloud 'be research questions. (However we did 
provide the user with features which made the program easy to interact with, 
such as graphics and an easy to use menu interface.) Serious investigation of 
teaching strategies before incorporating them into .sophisticated computer 
programs is rare in ITS research. This study serves as a possible model for 
such investigations. 

Human tutors seem to operate with a battery of tutoring strategies, 
many of which are relevant to specific contexts. They do not seem to use 
one general set of tutoring rules. We envision the bridging analogies strategy 
as one of many tutoring strategies that an intelligent computer tutor would 
have at its disposal, selecting each (or combinations of them) according to 
context applicability rules. We see these strategies as 'strong method" so- 
lutions to the computer tutoring probLm becau.se their appropriateness is 
context dependent, relying on information about the student, the discourse 
state, and the topic characteristics. Tutoring guidelines or strategies men- 
tioned in previous research are usually "weak methods" in that they are 
more generally applicable, but not as powerful as context sensitive strate- 
gies. Examples of weak method strategies are: "provide concrete examples 
of, as well a, descriptions of, concepts" (Burton & Drown 82); "provide im- 
mediate feedback for errors' (Anderson et. al. 8-1); and «, e t the student 
learn from mistakes" (Brown, Burton, & dcKIccr, 82). 



Ideally, the effectiveness of each of the strategics employable by a com- 
puter tutor should be researched. Our view i3 that it is desirable to tcsf, such 
strategies independently, first off line, and then in a prototype program. In 
this way, a great deal is known about these strategics before a large effort 
is expended in building an Intelligent Tutoring System based on them. Of 
course, further research is then needed to determine how to integrate several 
tutoring strategics into a single system. 

Key research questions 

At the onset of the research there were several key questions: 

• Can the strategy be reasonably simulated in a computer environment? 

• Can the strategy effectively remediate misconceptions? 

• Und*r what circumstances (i.e. what types of students, types of do- 
mains, discourse situations, etc.) does the strategy work? 

• What aspects of the strategy were most and least effective? 
Longer range questions that we are addressing are: 

• What other tutoring strategies might be effective where bridging analo- 
gies fails? 

• Is it possible to determine dynamically during a computer tutoring 
session when to use the bridging analogies strategy and when to switch 



to another strategy? 

The bridging analogies strategy presents the student with situations that 
bridge the conceptual gap between simple situations (those for which most 
students have correct intuitions) and difficult ones (those for which most 
students exhibit misconceptions), and encourages analogical thinking in an 
attempt to transfer the students' correct intuitions to a broader context. 
The strategy is designed to be applicable to topic areas with a qualitative, 
conceptual focus, where deep seated misconceptions exist, and where cor- 
rect intuitions (anchors) also exist. Basic Newtonian mechanics is one such 
domain. Real world experiences result in a wealth of intuitions about how 
the physical world works. Some of these are in accordance with scientific 
laws, and others in conflict with then. 

2 Background—Misconceptions in Science 

Research studie:, in recent years have investigated students' preconceptions 
about important mathematics and science concepts prior to formal instruc- 
tion. A surprising number and variety of misconceptions (incorrect pre- 
conceptions) have been uncovered in domains 8Uc h as electricity, optics, 
biology, statistics, and classical mechanics. Often these misconceptions are 
deep-seated and counterproductive to problem solving in the domain. Such 
incorrect beliefs are not limited to non-science students. Serious misconcep- 
tions are common among students taking calculus-Loscd physics courses and 
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students who arc proficient at quantitative problem solving using formulas. 
One implication for instruction is that the existence of those debilitating 
beliefs must be dealt with explicitly. They are not simply "erased" when 
conflicting and correct knowledge is conveyed to the student. Many of those 
misconceptions arc commonly held, so that attempts to define and remedy 
the most common of tlicm will benefit a large percentage of students. For 
instance, in a survey of 112 high school chemistry and biology students who 
had not taken physics, Clement (87) found that 76% of the students did not 
believe that a table exerts an upward force on (or pushes up on) a book 
resting on it (83% of these indicated a high confidence in their incorrect 
answer), a belief that is contrary to Newton's action-reaction law. 

Unfortunately, these misconceptions have proven to be quite resistant 
to remediation, at least via traditional instructional methods. Several stud- 
ies support this finding, the most comprehensive by Halloun and Hcstenes 
(1985). They administered tests of basic concepts in Newtonian mechan- 
ics (motion and its causes) to over 1000 college level introductory physics 
students over a three year period. The primary diagnostic instrument con- 
sistcd of multiple choice questions about very basic concepts, such as: "if 
a ball is shot out of a semi-circular tube, what is the shape of its path af- 
ter leaving the tube?" (the student was to choose between pictures of five 
path shapes). Average scores before instruction were "very low," (about 
45%) even for university physics courses where 80% are engineering majors, 
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and 8096 have taken calculus. Most importantly, the post-instruction scores 
showed little improvement (11 to 15%). Thin scant gain was remarkably 
independent of such variables as instructor, math ability, and final course 
grade. 

The literature cites several possible sources for students' misconceptions. 
White 06 Pri-deriksen (8G) note that misconceptions can result when the cog- 
nitive jumps required of a student while learning a new domain are too large 
or too under-specified. Learners can then fill in the gaps in unpredictable 
and uncontrollable ways. Such beliefs, acquired during instruction, might 
be called u mis-comprehcnsioiis.* Similarly, diSessa (1985) discusses how 
misconceptions can result when learners try to accommodate to new in- 
formation by incorrectly synthesizing existing fragmented and inconsistent 
pieces of knowledge. VanLchn (83) has constructed a detailed theory of how 
procedural knowledge is modified ("repaired") in an ad-hoc way when an 
impasse is encountered during problem solving. A similar mechanism may 
account for some (non-procedural) misconceptions. Claxt* n (85) points out 
that preconception;* can be founded on physical experiences ( u gut science") 
or on social experiences ('lay science*), and that these concepts can con- 
flict with the formal and abstract concepts of "school science." Students 
are often perfectly content with or unaware of their conflicting beliefs if the 
contexts ia which these beliefs arc used do not overlap, such as in everyday 
life vs. taking exams. 

8 
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Many misconceptions result from repeated attempts at comprehending 
real world phenomena. Some of these beliefs have been used again and again 
to successfully cope with or explain real world events. Therefore it !s not 
surprising that some misconceptions arc deep seated, and that some are quite 
common. Some domains, such as physics (McCIoskcy 83 and McDcrmott 84) 
and statistics (Tvcrsky ft Kahncman 74), arc particularly susceptible to the 
existence of misconceptions prior to any formal instruction. Interestingly, 
Clement (1982) has documented remarkable similarities between students' 
misconceptions and prc-Newtonian theories such as those of Galileo. In such 
domains, careful sequencing and explanation of new information is probably 
not sufficient, and existing misconceptions must be directly addressed with 
innovative instructional approaches. 

3 Description of the Bridging Analogies Tutor 

Clement & Brown (84) have developed a teaching strategy, called 'bridging 
analogies," which utilizes correct intuitions (which tl.ey call "conceptual an- 
chors") and appeals to students' analogical reasoning to help them extend 
their correct intuitions to target situations for which they have miscon- 
ceptions. After the student gives an incorrect analysis of a situation that 
indicates the existence of a common misconception (such as that rigid ob- 
jects don't exert forces), she is presented with an analogous anchor situation 
(such as a booh in a hand), which v/e will assume is indeed an anchor to 



the student .« The student ha, given contradictory answer, for the two sit- 
uations. Apparently she does not see them as analogous in terms of their 
relevant properties. The bridging analogies strategy attempts to bring the 
student to an understanding of this analogical relationship by presenting a 
sequence of intermediate analogies (called "bridging analogies'). At some 
point (or points) the student should be faced with considering two situa- 
tions for which she has given contradictory answers yet which she realizes 
are analogous. The cognitive conflict which results should motivate her to 
change her mind about the misconceived situation. 

The strategy employs Socratic type questioning to engage students in 
thought experiments: Decisions of whether to bridge are based on student 
responses, so that errors, rather than a pre-defined hierarchy of prerequi- 
site information, provide the catalyst for instmction. The strategy delays 
revealing new information about the subject matter and giving feedback 
about the correctness of student responses as long as possible. This encour- 
ages the construction of new ideas based on information which the student 
already has. This pedagogy, if taken to the extreme, is of course limited. 
The discourse can , sj r^ofar f rem the g 0 *| that consolidation becomes dif- 
Mn a cWoon, .it„«ic„ anchor, mu8 t be cho.en which are expected to be intuitive 
to the Verity of the eIttf . ,„ one . on . one , utoring( ^ ^ ^ ^ ^ & ^ 

of option, ,„ either case, previou, rcseirca mu ,t e.tab.ish which Nation, are u.eful 
anchor* for a given domain. 
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ficult, and some students become frustrated without a certain frequrncy of 
informative feedback. But withholding feedback can have many advantages 
as well. For most students, who are used to memorizing correct answers, 
immediate feedback am iuhibit deeper thought about conceptual issues. In 
instructional dialogues where feedback is withheld there can be more em- 
phasis ou the quality of the ideas themselves than on getting correct ajiitwers 
to questions. Students arc forced to evaluate their own knowledge and learn- 
ing process .. This instructional approach is based on the assumption that 
students must be actively engaged in constructing a new conception in or- 
der to replace their misconceptions. This research will help determine the 
effectiveness and limitations of this pedagogical style. 

For computer implementation, the strategy was formulated as a doubly 
recursive procedure which traverses a pre-defined network of example situ- 
ations according to student responses to questions. 7 The computer version 
of the strategy cannot respond to creative student insights, but it can tailor 
its presentation of examples personally for each student. We have used the 
program for different domains, but below we describe its implementation in 
teaching about the existence of opposing contact forces in static situations. 
This is the domain used for the majority of the research, and for which the 
most is known. 

T In this section we give an operational definition of the procedure. Sec Appendix A for 
a more precise description of the code implementing the strategy 

11 
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Figure 3: I Diagram of the bridgi ng analogies strategy 

bridge (the book on the spring) !s (apparently) remedied (sec Figure 3), and 
the anchor-to- bridge analogy is established. 

Once the bridge situation is understood and the ancljor-to-bridge anal- 
ogy established, wc wish to establish the bridgc-to-problcm analogy, thus 
constructing a path of unalogics or conceptual links connecting the anchor 
with the problem. The strategy compares the bridge (the book on the spring 
in this case) with the problem (the book cn the table) with the hope that 
the student will change her mind about the problem. If the student docs 
r,o we arc done (or done with the current recursive call of the algorithm). 
If the student docs not opt to change her mind about the problem, we arc 
back in our original condition, having a misconceived situation (the book on 
the table) and a correctly conceived one (t)ie book on the spring)— which 
this time is the bridging situation. The algorithm is called recursively to 
establish the bridgc-to-problcm analogy, beginning by finding a new bridge 
(in this case the book on a flexible board) and so on. 

One perspective on the strategy is that is keeps splitting the concep- 
tual difference between the simplest misunderstood situation and the most 
difficult understood situation. Eventually it provides the student with an 
intuitively valid analogical path from the original anchor to the original prob- 
lem, such that the gaps between neighboring situations are close enough for 
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the student to perceive the analogy. Splitting the difference at each step is 
more efficient than walking the student through every smallest step of the 
entire set of example situations linking the problem with the anchor. The 
dark line in Figure 1 shows one student's traversal through the network. 
Since the strategy is a m ; ^ lie out", the path shows the order that situations 
were understood, rather than the order in which they were introduced. 

Hints for example situations 
Available for every example situation are: 

• a description of the situation 

• optionally, a "detailed description" (as described below) 

• a question with multiple choice answers 

• a hint or explanation of the correct answer 

• and, optionally, a pointer to a graphics file for the picture. 

The system does not distinguish between hints and explanations. The 
ones used in the static forces network varied along a continuum from weak 
hints to strong hints to correct answer descriptions. There arc two circum- 
stances where the hint or correct explanation is given to the student. One 
is when the student, after a comparison, opts to change her mind about 
an anchor and changes it from right to wrong. At this point we have no 

17 



more graceful method for dealing with these "regressions*. They occur very 
rarely in our tests so far. 

The second condition warranting a hint or explanation (i.e. the student 
gets immediate feedback after a wrong answer) is when there is no bridge 
in the network between the current problem and anchor. The hint is given, 

often suggesting that the two situations arc analogous, and the student is 

asked the problem question again. 

Confidence checking 

In order to simplify the description of the bridging analogies strategy, we 
have thus f ft r neglected an important feature: confidence checking. 

A severe drawback to computer tutoring, as compared with human tu- 
toring, b i* inability , 0 ascertain the types of affective information about 
the student that are evident to human tutors f.om facial expressions, vo- 
cal inflections, etc. Information about students' certainty level, confusion, 
boredom, excitement, etc., are crucial to effective human tutors. 

Human tutoring studios at the University of Massachusetts have used 
"how sure are you" or, alternately; "how much .ense does the answer make- 
scales to gather additional information about the strengths of anchors and 
the effectiveness of the bridging strategy in changing students' beliefs. Cor- 
rowing on this idea, the computer tutor asks the student to rate her confi- 
dence in her answer on a five point scale after each question (as shown in 
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Figure 2). The program combines this information with information about 
the correctness of the answers in deciding whether to ask for comparisons, 
and whether to bridge (see the section on data structures and algorithm for 
details). 

As mentioned above, the strategy postpones any type of feedback con- 
cerning the correctness of a student's answer as long as possible. 12 The 
algorithm is sufficiently complex so that the presentation of situations and 
comparison questions appear random to the user. Depending on the pro- 
gram's internal state in the recursive procedure calls, a new situation or 
comparison could be given after cither a right or wrong student response. 
Therefore, the student usually docs not know why she is presented with a 
new example or ai-kcd to compare situations. This tends to keep the stu- 
dent focused and thinking about the current physics question, rather than 
second- guessing the tutor's intentions. The student tends to answer accord- 
ing to what she believes, rather than what she thinks the tutor wants her to 
belic\c (as can happen when students have an attitude of getting the most 
"points" rather than an interest in thinking and learning). 

Since confidence information is important, the comparison allows stu- 
dents to change their confidence about a previous answer as well as the 
"This strategy is in some aspects etf reme. The reader should keep in mind that we 
are conducting research on specific strategies in isolation, to rssess their efTectiveneas in 
limited instructional contexts. 
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answer itself. We have noted in some computer tutoring sessions that ask- 
ing students to commit to a confidence rating causes them to put additional 
cognitive effort into visualiziug and analyzing the example situations. 

Topics 

Example situations are grouped into "topics" corresponding to the physics 
concepts to be taught. The topic's grain size is relatively small, and each 
topic has a network of examples. The topic for the network shown in this 
paper is "existence of contact forces in the vertical direction between two 
stationary objects." For each topic area the goal of the tutor is to help the 
student understand the qualitative physics of a key "target situation" which 
exemplifies an expert application of the concept. If the student answers 
incorrectly about a target situation, then the bridging analogies strategy is 
invoked. The topics are given in sequence, and if there is any overlap be- 
tween example situations in the topic areas, the program takes into account 
previous answers about the situations. In this paper we describe results for 
only the topic network shown in Figure 1. 

Designing the example situation network 

The network design can be described in two ways: in terms of example 
difficulty, or in term- of example features. We will discuss each below. It 
should be kept in mind, though, that the final version of the network we 

20 
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used was influenced by empirical results from human tutoring studies. 

Traversal along the extremities of the net will touch each example sir- 
uation, yielding a sequence that can be interpreted as, starting with the 
original anchor, progressing from easier to harder problems. Each bridge is 
intended to split the conceptual distance between two other situations. The 
bridging analogies strategy is seen in this light as efficiently searching the 
space of examples using a divide and conquer approach. The sequence of 
examples between the current problem and the current anchor is divided at 
the bridge specified by tlu* network. The network thus encodes the peda- 
gogical knowledge of estimated conceptual distance (as opposed to distance 
in a linear sequence of examples, where the third would be halfway between 
the first and fifth). 

A second perspective describing the network involves feature dimensions 
of the examples, and is an equally valid description for the book on the tabic 
network we have been using. Research has uncovered several reasons for 
students 1 beliefs that the table can not push up on the book, but a hand can 
push up on a book. Each reason involves an attribute which distinguishes the 
two examples. Reasons include: a person is doing something, but the table 
is just in the way; a person's arm can move to adjust to the book's weight, 
but the table cannot; an arm's purpose is to exert forces, but a table's is not. 
Thus the student sees one or several features of objects as being rclcvrmt to 
their ability to exert forces, all of which are irrelevant from a physicist's 
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standpoint. The network factors out these possible features incrementally 
by providing; example situations with different sets of relevant and irrelevant 
features. For example, the spring can move and is designed 'o exert a force, 
but can not decide on its own to volition to do so. 13 

Other features of the tutor 

The tutor is written in PASCAL and runs on an MM PC. As was mentioned 
above, the domain specific information and inter-example links can be easily 
modified, or new domains created from scratch, by straightforward word 
processing of text files. However, it is not easy to design effective networks of 
anchors and bridges. This requires research iterations with human subjects. 

Figure 2 shows examples of the graphic pictures that accompany the 
descriptions of the example situations. Created using the PC-Paint (trade- 
mark of Mouse Systems) program, their inclusion is optional. 

All student input is through menu selections. The use of menu-driven 
options, and of multiple choice answers, allows us to bypass difficult prob- 
lems of language recognition involved in interpreting student input. Multiple 
choice answers can include reasons for the answers and "distractor" items 
winch catch common wiong answers and misconceptions. Usually, reasons 
for a student s answer arc just as important as the correctness of the answer. 

Example situations can have two levels of description associated with 
la Si»e the section on Implications for a more formal approach. 
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them. After being presented with a description of a new situation, the 
student is asked if she wants a more detailed description. The detailed 
description attempts to address possible areas of confusion which might 
arise. 

4 Formative Evaluation of the Tutor 

The purposes of this phase of evaluation are, as mentioned, to determine 
what aspects of the simulated strategy are effective, and in what instruc- 
tional contexts. We are also interested in information leading to alternative 
computer tutoring strategics to be used where bridging is not effective. 

The first working version of the tutor was quite primitive. We were not 
sure whether a machine simulation of the human strategy would be so dif- 
ferent as to make previous results concerning the usefulness of the strategy 
irrelevant. A pilot study was conducted with nn interviewer showing the 
subjects picture cards corresponding to the text on the computer screen. 
Evaluation of the videotaped student interaction* confirmed that an auto- 
mated version had promise, and we began a second design cycle, which is the 
focus of this paper. The pilot study provided information leading to several 
modifications to the program. The user interface was revised, the format 
of the comparison question was changed, graphics capabilities were added, 
and several nodes were added to the book on the table example network. 
To locate subjects suitable for an interview study of the revised tutor, a 
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pretest was administered to students in several university classes. A total of 
180 students were given a pretest with four example situations, questions, 
and confidence ratings. None of the questions were the name as the ones 
in the tutor's network. One situation was similar to the original problem, 
involving contact forces on a medium sized object resting on an inflexible 
object. Another was similar to the initial anchor, where one object was 
held in a hand. 81 percent of the students answered the suspected anchor 
correctly, and G8 percent answered the suspected problem incorrectly. OS 
percent got both the problem wrong and the anchor light. These figures are 
in rough agreement with the i<Mults of previous studio. 

Of the students who took the pretest, 53 percent volunteered to be sub- 
jects for a one hour video taped interview study, for which they would be 
remunerated 5 dollars if chosen. H 

Twenty five students were selected for interviews. The selection of stu- 
dents was biased toward those for whom we thought the strategy, and the 
network used, were most appropriate. Those who answered the problem in- 
correctly with high confidence and the anchor correctly with high confidence 
were given preference. Three of the sessions involved pairs of students usins 
the tutor. There were technical problems (including power outage*), and 

several students unexpectedly answered the original problem correctly with 
u The distribution of correct answer* for the two aituationa did not differ significantly 

between those who volunteered and thoie who didn't. 
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high confidence- right from the start-in which case the bridging analogies 
strategy is not invoked at all (or needed). In the end we had a total of 15 
sessions with useful data from the book on the table network. 

Summary of the results 

Several types 0 f data were recorded (see Appendix B for a description of how 
the data were analyzed). A„ interviewer, who was present to elicit thought 
verbalization and answer student questions about using the program, " 
took notes during the sessions. We have completed a fin* pass analysis of 
the video taped recordings. See Schultz et. al. (87) for a protocol analysis 
report. The program stores a trace of each session, and this information has 
been summarized and analyzed. Following are 8 ome observations: 

• At the end cf the sessions all but one of the 15 subjects had a high 
confidence in the correct answer to the original problem situation, and 
most of their verbal reasons indicated that the correct answer made 
sense to then, at an intuitive level. Nine of them started the session 
with a high confidence in the wrong answer to the original problem 
(The rest started with a low confidence in the wrong answer to the 
original problem.) 
"The current ver.cn of the program i, „„ t robun enollgh to be ^ 
supervision. 
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a Most of the subjects needed to pass through a node which described all 
matter as being composed of molecules connected by springy bonds. 
Since we designed the network, we have come to categorize this type 
of example as a "causal model* (Drown & Clement 87), rather than a 
bridging analogy. It is not clear whether their misconception concern- 
ing the book on the table could have been remedied for tb;se subjects 
without the use of a causal model. 

• The analogical comparisons alone successfully instigated a change *n 
belief at some point in the network for 50 percent of the subjects. 
This suggests that bridging and analogical thinking are effective in- 
structional strategies for many students at least locally (i.e. if the 
analogy is not too distant), but perhaps not when used exclusively. 

• Nine of the subjects needed at least one explanation or hint nt some 
point in the .session. This could indicate that no bridge was available 
where the student needed one, or that the bridging analogies strategy 
is not powerful enough to be mod exclusively in computer tutoring in 
these types of domains. 

• There were only two instances of "regression,'* where a student, after a 
comparison question, attempted to change her mind about an anchor 
from right to wrong. Also, there was only one case where a student 
did not answer correctly about the original anchor with medium or 
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high confidence (in which case the program had to explain the answer 
to the original anchor). These facts indicate that the anchor used was 
an elective starting place for remediating the primary misconception. 

On the average, 7.5 example situations were needed to bring the sul>- 
jeet to a successful conclusion of the session. This supports the need 
for many nodes in the networks, and indicates that the original anchor 
and problem were indeed "distant" analogies. 

Most of t he instances of giving explanations or strong hints occurred in 
the area of the molecular model example. This and other observations 
from the experiment will provide useful and detailed information con- 
cerning where students have the least and most difficulty, and where 
in the example network more bridges need to be added. 

Only one of the 15 subjects spent any time in the area of the network 
between the book on the spring and the book on the hand, suggesting 
that the book on the spring would serve equally well as the original 
anchor. (This finding is in agreement with the results of previous 
experiments by Clement and Brcwn). 

Three of the bessions involved pairs of subjects using the tutor. They 
were asked to discuss their beliefs about the questions and cornc to 
an agreement about an answer. These sessions were quite animated. 
There was a high degree of motivation, verbalization, and serious re- 
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flection about the problems. Only rarely did the inter/fewer need to 
intervene and ask for verbalization of thoughts or reasons for answers. 
These interactions arc encouraging, and we plan to do further studies 
using pairs. 

• Students' comments about the program, elicited at the conclusion of 
the session, were in general favorable regarding the effectiveness of the 
learning experience and their enjoyment in using the tutor. 

Defining instructional contexts 

We are starting to ascertain the instructional contexts in which the strategy 
is and is not effective. We have noticed four categories that student behavior 
patterns seem to fall into. The fir.t, for which the strategy is most effective, 
includes students for whom introducing bridges and encouraging analogical 
comparisons lead to changing belief to a confident correct understanding of 
a misconceived situation. They needed few or no hints, and were usually 
confident in both their correct and incorrect answers. The second group 
often needed to be given strong hints or explanations of correct answers.. 
This group is perhaps not proficient at analogical thinking, or was not con- 
vinced by the particular example situations which they were given. A third 
group is characterized by net making much progress toward understanding 
the target problem (the book on the table) until the causal model was .n- 
troduced, which then led to a complete reversal in their intuitions regarding 
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the target. It iy not clear whether thinking about the previously presented 
bridging analogies contributed to this change, or whether giving the causal 
model alone would have been just as effective. The fourth group is char- 
acterized by indicating a low confidence for all answers given (both correct 
and incorrect). Our impression of this group is that they were not men- 
tally engaged in the thought experiments about the physics of the example 
situations, or had low self esteem. 

This grouping is not a partition of the subjects into four distinct groups. 
Some students exhibited characteristics of more than one group. We have 
done only preliminary work on how i computer tutor could t'se the infor- 
mation at its disposal to determine which of these four behavioral contexts 
a student is in, and what alternate tutoring strategy might be invoked. For 
example, a student indicating low confidence on most o r her answers might 
occasionally be given positive feedback on correct anchor cases. A student 
who is only changing her mind as a result of strong hints might be presented 
will) a causal model earlier than she otherwise would. A student for whom 
the strategy can. 1 find an anchor might be given remedial instruction or a 
didactic explanation arguing against the misconception. We expect that the 
context and appropriate strategy will fluctuate within a tutoring session. 

In summary, there is evidence that the strategy w^'ks effectively for 
many students. Wc are encouraged by the levels of cognitive involvement 
in the thought experiments which the subjects exhibited, especially in con- 
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traat with the more passive role that student, tend to take in traditional 
expk ,tion oriented method, of teaching, or in computer aided instruc- 
tion that provides imm ediate feedback for wrong answer,. The formative 
evaluation provided useful information for t | lC improvement of the specific 
example network used, for improving the bridging analogies strategy, and 
for implementing alternative tutoring strategies. 

5 Implications for a more intelligent tutoring sys- 
tem 

The current implementation of the bridging analogies strategy has served 
well as a research vehicle. Video taped tests showed that in certain situations 
the strategy engages students and brings about significant changes in belief. 
But the tests also showed that the current implementation i, not flexible or 
complete enough to act a.s a stand-alone tutor (at least in the domain used). 
The knowledge representation and decision control power of AI technology 
may provide the leverage needed to design a robust tutor which utilizes 
bridging analogies as a central strategy. 

The tests indicate that alternative strategies, such as presenting causal 
models and giving leadrg questions, may be necessary as well. Also, with no 
model of the student's beliefs, the program had to blindly present example 
situations until one of the stud ent's "critical points" was reached.'" 
'«We would li kt to avoid printing ,„ perfluouj examp|cj ,„ thf ^ ^ ^ 
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Below arc three recommendations for more intelligent versions of com- 
puter tutors that automate the bridging analogies strategy 17 : 

• As a tutoring session progress- 3, construct a model of the student's 
beliefs to be used to make tutoring decisions. Data from answers, con- 
fidence ratings, reasons for answers, changes of mind, etc. contribute 
to the student model. 

• Select the appropriate bridge between the current problem and anchor 
(currently done with a lookup table specified by the instructional de- 
signer) via an intelligent inference mechanism that takes into account 
key features of the problem and anchor, and information from the 
student model. 

• Incorporate alternative tutoring strategies, such as using causal mod- 
els, extreme cases, count er examples, and leading questions, to be used 

advocate trying to move directly to crit.cil point, (turning for the moment that tl.ii 
would be possible) because the critical point may represent a cumulative effect fiom several 
ex&mplct. 

,7 Two of us (Murray and Schultz) are in the process of redesigning the tutor for an arti- 
ficial intelligence workstation using AI programming technologies. The research described 
in this paper was conducted under the auspices of the Cognitive Processes Research Group 
- the University of Massachusetts. The re-designed tutor will be written under the aus- 
pices of the Intelligent Tutoring System, Group, ;„ the UMa.s Department of Computer 
and Information Science. 
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questions, but maximum learning leverage would of course be obtained in 
a system using both rich environments and intelligent discourse. Also, we 
foresee other "strong method" tutoring strategics being developed to teach 
science. Strategies arc needed to teach factual knowledge, problem solv- 
ing skills, and scientific inquiry skills, and to guide students' exploration of 
learning environments that provide new types of experiences. 

Below we summarize what we believe are the distinctive (though not 
unique) features of this research: 

1. Relying on previous cognitive studies of a tutoring strategy. 

2. Testing the simulated strategy early on, before large cITort is expended 
in coding an elaborate tutor. 

3. Incorporating multiple evaluation-redesign stages using interviewed 
subjects before a final robust system is built. 

4. Being explicit about the tutoring strategy used. 

5. Researching the strengths and limitations of a highly context sensitive 
("strong method") strategy in isoiaticn. 

6. Focusing on a single strategy which is ultimately intended to be used 
in concert with other strategies in an intelligent tutoring system. 

7. Focusing on changing students' intuitive beliefs and teaching qualita- 
tive conceptual knowledge, as opposed to teaching or debugging skill 
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knowledge. 



8. Using of analogies to extend the locus of applicability of existing intu- 
itions. 



0- Our main instructional leverage cones from computer generate ! So- 
cratic type dialogue, as compared with computer simulation, a-, is 
common in other science ITS's. 

10. Focusing research on the process of tutorial interaction, rather than 
on task analysis. 

7 Conclusions 

The discouraging results of recent broad-based education studies (The Na- 
tional Commission on Excellence in Education, 1983, and The National 
Science Board Commission on Pre-collcge Education in Math, Sconce, and 
Technology, 1983) have spurred concerns about the quality of science educa- 
tion. Educational researchers arc actively studying the process of learning 
science and are searching for innovative instructional approaches Some of 
this research deals with how students' preconceptions .nfluence their learn- 
ing. Research has shown that misconceptions in the sciences are widespread, 
debilitating, and resistant to change. Cognitive studies have categorized 
many of these misconceptions and are experimenting with techniques for 
dealing with them. 
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A tutoring strategy found to be useful in remediating physics miscon- 
ceptions in written explanation and onc-on-one tutoring studies has been 
simulated in a computer tutor, which we call the bridging analogies tutor. 
Preliminary results from a formative evaluation involving videotaped ses- 
sions with the tutor in the domain of Newton s third law are encouraging. 
The program engaged subjects in thought experiments at an intuitive level, 
and resulted in changes in belief about misconceived problem situations. 
The evaluation resulted in many specific implications for computer assisted 
instruction in the domain used, and for using the bridging analogies strategy 
in general. 

Though conclusions about t he general effectiveness and applicability of 
the hridgirg analogies strategy must be considered cautiously, we believe 
that the strategy shows promise for some instructional contexts, and we 
plan to continue the research. Taking into consideration *hc results of the 
evaluation, we have given suggestions for a rc-design of the tutor, using arti- 
ficial intelligence programming technology, to incorporate student modeling, 
intelligent example selection, and alternate tutoring strategics. 

Further work is needed to codify human tutoring strategies, test their 
effectiveness in computer learning environments, and determine how to or- 
chestrate the different strategies in computer tutoring sessions. We have 
relied on previous cognitive studies of misconceptions and tutoring strate- 
gies in to formulate our ideas. This research exemplifies the importance we 
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Figure 4: An Algorithm for the Bridging Analo gics Strategy 



APPENDICES 



A Program data structures and algorithm 

Domain information is stored in text files, making the program which drives 
the tutor independent of domain information. There are three domain files. 
The Topic File determines the sequence of topics to be presented (ibis paper 
discusses only one of the topics), and specifies which of the examples in 
the topic's network will serve as the original anchor and original problem 
situations. The Network File contains triplets of numbers, each number 
representing an example situation node. Each triplet defines a unique bridge 
between two other nodes, and defines the network as in Figure I. The tabic 
thus defined is searched when the program bridges between a problem and 
an anchor. "Wild cards" arc incorporated, allowing the designer to specify 
the use of a certain bridge whenever a specified node is the problem (or 
anchor), regardless of the anchor (or problem). 

The Situation File contains text for the example descriptions, detailed 
example descriptions, questions, multiple choice answers, correct answer, 
and a hint or explanation for the correct answer. This information is or- 
ganized using keywords, and is parsed and loaded into PASCAL record 
structures when the program is executed. These record structures also keep 
track of the student answers and confidence ratings for each node. 

The bridging analogies strategy is a doubly recursive algorithm for travers- 
ing the network (the network is topologically equivalent to a binary tree). 
The algorithm has been described operationally in Section 3. It appears 
in pscildo* code in Figure 1. The code shows what parameters are used to 
make decisions in the procedure and where in the procedure the recursion 
occurs. 

The procedure Teach-Topic is called itcrativcly for each topic in the 
Topic File. The Bridge procedure contains the algorithm for traversing the 
network. The program is capable of "finding 3 the original anchor., but this 
capability was switched off (and the original anchor is given in the Topic 
file, as described above) in the test runs we report in this paper. It finds 
an original anchor by presenting the student with all the situations linked 
to the original anchor in order of increasing simplicity, until the student 
answers one correctly with high confidence. For large networks this may 
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be more efficient than assuming that the best original anchor is the very 
simplest example. 

The algorithm is relatively simple, yet due to its recursive nature, and 
the fact that the confidence level is used differently in sections 2 and 3 of the 
Bridge procedure, it is quite difficult for a student using it to distinguish a 
predictable pattern. The presentation of examples, hints, and comparisons 
seems random to one not familiar with the algorithm, Recall that we do not 
explicitly tell students whether they arc correct on any questions, so it h 
to great advantage that the student cannot (or rarely has in our tests thus 
far) determine the correctness of an answer merely from the type of action 
taken by the tutor, 



41 



39 



Figure 5: Sample section of a Trace file 



Figure 6: Diagramatic Representation of the Program Trace 



B Data Analysis 

There were three sources of data: notes taken by the interviewer during 
the sessions, a computer trace of the tutor's actions and uaer responses, 
and videotaped recordings of the sessions. Analysis of the videotapes is 
described in Scluiltz ct. al. (87). Here we will outline our method for 
analyzing the computer traces. The trace records all important cvcnt3 in a 
readable form, as in Figure f>. This information was transferred (by hand) 
to a diagramatic representation, as in Figure 6. 18 Circled nodes arc those 
visited. Next to each circled node the quality of the answer (ok or wrong) 
and the confidence rating (from 1 -blind guess to 5- I'm sure) arc recorded. 
Single line arrows go from problems or anchors to the next bridge. Douhb 
lined arrows go from bridges to problems, and show where the subjects 
changed their mind from wrong (or right with low confidence) to correct 
(or medium to high confidence) for the problem pointed to. Circular lined 
arrows which connect a node with itself represent changing their mind on 
the question most recently aiked, or, if an "IP is shown, they were given 
a hint and n.-<krd flic question again (as would happen if no bridge were 
available between two nodes). Not represented in the diagram, but easily 
inferred from the information in it, arc the places where the tutor asks for 
a comparison between the most recently given situation and a previous one. 
Ilccall that after eacli such comparison the student is asked if she wfJits to 
change her mind on either or both situations. 10 

Data analysis is greatly enhanced by the diagramatic representation. 
The subjects' path through the example network, their changes of mind, 
and where hints were needed, arc highly visible. We can easily sec regions of 
the net where the subjects needed the most bridges. From this we can infer 
which attributes of the original problem contributed to their misconception. 

For each subject's network, the following data were determined: the 

'•Thii figure ihows the entire network used. The portion to the left, between the book 
^ the table and a fly on the road, is ignored in other parts of the paper for simplicity. 

With seme programming effort (moat of it for graphics) the conversion from trace to 
diagram could be automated, but wc choce not to do bo. 
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number of nodes visited, the number of hints given, the number of changes 
in mind, the total change in the original problem from the beginning to the 
end of the session, the "critical" changes in mind (defined below), and the 
"focal regions" (defined below). A summary of this information ia fcivcu in 
the Results section of this paper. 

The network was divided intc four "focal regions," each addressing a 
different key attribute of the target problem. The "focal regions" data 
specifics in which regions the subject visited two or more nodes. 

"Critical" changes give a more reasonable estimate of the student's think- 
ing than the total changes of n\ind, and are defined as follows. Often, one 
realization on the part of the subject leads to changes of mind for several 
situations. For example, a student who h being shown a hand on a table 
situation (node 10) is four recursive levels deep in the algorithm, hp.ving 
answered incorrectly questions about a book on a table, a book on a flexible 
board, and many books on a flexible board. Considering the hand on the 
board may be a critical analogy for this student. After answering it cor- 
rectly he may realize that several of his previous anscrs should be changed. 
As the rccuisive algorithm unwinds (sec Fijjure 4), he would be given three 
comparison questions in a row, and opt to change his previous answer for 
each of tiles'? to correct with high confidence. When the subject changes his 
mind in a "chain" like this, all changes can be the result of a single change 
in belief. "Critical changes" are the firbt changes in any chains of two of 
more change*, plus all the change** that arc not in a chain. 
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Network of example situations for ™«t 

ns ror existence of normal forces 



42 F«„. d_ 



Imagine a iiu'tliuiii >'uc textbook resting un a 
dining ruum tabic. 



While the book is resiing there the tabic: 

A. IS exerting a force up oo the book 

U IS N^T exerting a force up on the book 

Please rate yuur confidence io thia answer: 
'Mind Que 33 ' 'not very con}' 'iotnewhat conf 
'fairly conf Ttn ture * 

Imagine that you are holding a tcxlbuuk in yuur 
hand. 

While the book is reeling there your hand: 

A. IS exerting a furce up on the book 

U. IS NOT exerting a force up uu the buok 

ugJr, give a n answer and '.oufidence ** - ) 




1-or **a book io your hand," you uaiJ: 

your hand IS exerting a force up oo 'lie book 

(with high confidence). 

Uut for "the book on the table," you a aid. 

the table IS NOT exerting a force up on the book 

(with fair confidence). 

Kxplain why thc^e anawera are different. 

•••Do you want to change your mind on either or 
both of them? ••• 

Figure 2: A sample dialogue showing the original problem and anchor 
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Bridging Analogies Strategy 




• * 



Procedure Teach-Topic (orlginal-problea, orlginal-a-chor) 
Descrlbe-and-qnestion(orlglnal-probi.»n) 
Descrlbe-and-qnestion (original-anchor) 

Coapare-and-allow-change(origlnal-problea. original-anchor) 

IF anchor wrong or confldenceO 
THEN Hint (original-anchor) 

IF problea is right with confidence^ 
THEN RETURN {no nlsconception} 
ELSE Brldge(orlginal-problea ( original-anchor) 

Procedure Bridge (problea, anchor) 

(The anchor was previously answered correctly with 
confidence >2 and the problem was previously answered 
Incorrectly or with confidence <3 } 

(1. find the bridge} 

Find-a-bridge-between (problex, anchor) 
IF no bridge exists 

THEH Hint (problea) AHD RETURH 
Describe-and-Queation (bridge) 

{2. establish bridge-to-anchor analogy:} 

IF bridge wrong or confidence^ 

THEH Comparc-and-allow-change (bridge, anchor) 

IF anchor changed to wrong or confldenceO 
THEN Hint (anchor) 

IF bridge wrong or confldenceO 

THEH brldgearidge, anchor) {first recursive call} 

{3. establish problea-to-brldje analogy:} 
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Conpare-and-allo*-change(proble«, bridge) 

IF problem Is wrong or confidenceO 

IHEfl bridge(proble., bridge) {second recursive call} 

{4. At this point the problem .hoold hive been answered correctly) 
RETURN 

Procedure Hint (an-exaaple) 

{iteratively gives the hint afid re-aaha the exaaple's 
question until the student answers correctly, or antil 
given three hints in a row, in which case the student 
is told exactly that to enter as the answer(l)} 

(Hole: Confidence ratings fron 1 to 6 are for 'blind guess' 
to T» sure' as in Figure 2.) 
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Figure 5: Sample oopHa« _ » 

START SESSION: 
program version: 11/18/86.1 
Name: Willard 0. Oz 
Session ID: 292 

Topic: B00KTABLE 

0-prob, 0-anch, 0-eub: 13 0 8 



ASK 1 book on the table 
Response: B (not OK) Confidence: 5 

ASK 2 many books in hand 
Response: A (OK) Confidence: 5 

-BRIDGE: P 1 A 2 8 3 11:50:35 
ASK 3 one book in hand 
Response: A (OK) Confidence: 5 

COMPARISON: 2 and 1; no change 

•BRIDGE: P A B 11:50:47 

ASK 4 book on spring 

Response: b (Not OK) Confidence: 3 

Comparison: 4 and 3; no change 

•BRIDGE: P 4 A 3 B 5 11:51:17 
ASK 5 hand on book on spring 
Response: A (OK) Confidence: 2 

Comparison: 5 and 3; change 5 

ASK 5 hand on book on a ing 
Response: A (OK) Confidence: 4 

Comparison? 5 and 4; change 4 

ASK 4 book on spring 
Response: A (OK) Confidence: 4 

Comparison: 4 and 1; no change 

•BRIDGE: P 1 A 4 B 8 11:53:03 

ASK 8 book on board 

Response: A (OK) Confidence: 4 



race tile 
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ur. 6 Di.flr.-Uc r.pr...nt.tion of th. progr.. tr.c. 






viaited 

anaver quality 
-confidence rating (1 to 5) 

to next bridge 

changed wind from wrong to right 
changed »inJ on questions Juet i«ked 
hint given 
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